Occlusions in motion processing

The instantaneous identification of independently moving objects in a video is particularly difficult if the camera itself is moving, since the motion field on the image is created by the combined effect of camera motion (egomotion), structure (depth), and the independent motion of scene entities. For a camera with a restricted field of view undergoing a small motion between frames, there exists in general a set of 3D motions (translation, rotation) compatible with the observed flow field, even if only small amounts of noise are present. We address the instantaneous motion segmentation problem in the presence of this ambiguity in obtaining exact solutions for interframe 3D motion. We show that moving objects can be classified based on the amount of information required for their detection: if separable clusters of solutions exist, motion based clustering alone will suffice for detecting the simplest class of moving objects. If only a single cluster is found, the positive depth (structure) constraint can help us identify a second class of moving objects. A more difficult third class of moving objects is found by detecting conflicts between occlusions and structure from motion. Occlusions can not only reduce the ambiguity in 3D motion estimation but also help us identify an important class of moving objects. We underscore the observation that occlusions must not merely be identified, they must also be ‘filled’, so that ordinal depth may be deduced.